Visual Object Tracking in Challenging Situations using a Bayesian Perspective = Seguimiento visual de objetos en situaciones complejas mediante un enfoque bayesiano

The increasing availability of powerful computers and high quality video cameras has allowed the proliferation of video based systems, which perform tasks such as vehicle navigation, traffic monitoring, surveillance, etc. A fundamental component in these systems is the visual tracking of objects of interest, whose main goal is to estimate the object trajectories in a video sequence. For this purpose, two different kinds of information are used: detections obtained by the analysis of video streams and prior knowledge about the object dynamics. However, this information is usually corrupted by the sensor noise, the varying object appearance, illumination changes, cluttered backgrounds, object interactions, and the camera ego-motion. While there exist reliable algorithms for tracking a single object in constrained scenarios, the object tracking is still a challenge in uncontrolled situations involving multiple interacting objects, heavily-cluttered scenarios, moving cameras, and complex object dynamics. In this dissertation, the aim has been to develop efficient tracking solutions for two complex tracking situations. The first one consists in tracking a single object in heavily-cluttered scenarios with a moving camera. To address this situation, an advanced Bayesian framework has been designed that jointly models the object and camera dynamics. As a result, it can predict satisfactorily the evolution of a tracked object in situations with high uncertainty about the object location. In addition, the algorithm is robust to the background clutter, avoiding tracking failures due to the presence of similar objects. The other tracking situation focuses on the interactions of multiple objects with a static camera. To tackle this problem, a novel Bayesian model has been developed, which manages complex object interactions by means of an advanced object dynamic model that is sensitive to object interactions. This is achieved by inferring the occlusion events, which in turn trigger different choices of object motion. The tracking algorithm can also handle false and missing detections through a probabilistic data association stage. Excellent results have been obtained using publicly available databases, proving the efficiency of the developed Bayesian tracking models. La creciente disponibilidad de potentes ordenadores y cámaras de alta calidad ha permitido la proliferación de sistemas basados en vídeo para la navegación de vehículos, la monitorización del tráfico, la vídeo-vigilancia, etc. Una parte esencial en estos sistemas es seguimiento de objetos, siendo su principal objetivo la estimación de las trayectorias en secuencias de vídeo. Para tal fin, se usan dos tipos de información: las detecciones obtenidas del análisis del vídeo y el conocimiento a priori de la dinámica de los objetos. Sin embargo, esta información suele estar distorsionada por el ruido del sensor, la variación en la apariencia de los objetos, los cambios de iluminación, escenas muy estructuradas y el movimiento de la cámara. Mientras existen algoritmos fiables para el seguimiento de un único objeto en escenarios controlados, el seguimiento es todavía un reto en situaciones no restringidas caracterizadas por múltiples objetos interactivos, escenarios muy estructurados y cámaras en movimiento. En esta tesis, el objetivo ha sido el desarrollo de algoritmos de seguimientos eficientes para dos situaciones especialmente complicadas. La primera consiste en seguir un único objeto en escenas muy estructuradas con una cámara en movimiento. Para tratar esta situación, se ha diseñado un sofisticado marco bayesiano que modela conjuntamente la dinámica de la cámara y el objeto. Esto permite predecir satisfactoriamente la evolución de la posición de los objetos en situaciones de gran incertidumbre. Además, el algoritmo es robusto a fondos estructurados, evitando errores por la presencia de objetos similares. La otra situación considerada se ha centrado en las interacciones de objetos con una cámara estática. Para tal fin, se ha desarrollado un novedoso modelo bayesiano que gestiona las interacciones mediante un avanzado modelo dinámico. éste se basa en la inferencia de oclusiones entre objetos, las cuales a su vez dan lugar a diferentes tipos de movimiento de objeto. El algoritmo es también capaz de gestionar detecciones pérdidas y falsas detecciones a través de una etapa de asociación de datos probabilística. Se han obtenido excelentes resultados en diversas bases de datos, lo que prueba la eficiencia de los modelos bayesianos de seguimiento desarrollados

An Efficient Multiple Object Detection and Tracking Framework for Automatic Counting and Video Surveillance Applications

Automatic visual object counting and video surveillance have important applications for home and business environments, such as security and management of access points. However, in order to obtain a satisfactory performance these technologies need professional and expensive hardware, complex installations and setups, and the supervision of qualified workers. In this paper, an efficient visual detection and tracking framework is proposed for the tasks of object counting and surveillance, which meets the requirements of the consumer electronics: off-the-shelf equipment, easy installation and configuration, and unsupervised working conditions. This is accomplished by a novel Bayesian tracking model that can manage multimodal distributions without explicitly computing the association between tracked objects and detections. In addition, it is robust to erroneous, distorted and missing detections. The proposed algorithm is compared with a recent work, also focused on consumer electronics, proving its superior performance

Robust Tracking in Aerial Imagery Based on an Ego-Motion Bayesian Model

A novel strategy for object tracking in aerial imagery is presented, which is able to deal with complex situations where the camera ego-motion cannot be reliably estimated due to the aperture problem (related to low structured scenes), the strong ego-motion, and/or the presence of independent moving objects. The proposed algorithm is based on a complex modeling of the dynamic information, which simulates both the object and the camera dynamics to predict the putative object locations. In this model, the camera dynamics is probabilistically formulated as a weighted set of affine transformations that represent possible camera ego-motions. This dynamic model is used in a Particle Filter framework to distinguish the actual object location among the multiple candidates, that result from complex cluttered backgrounds, and the presence of several moving objects. The proposed strategy has been tested with the aerial FLIR AMCOM dataset, and its performance has been also compared with other tracking techniques to demonstrate its efficiency

Bayesian Visual Surveillance, a Model for Detecting and Tracking a variable number of moving objects

An automatic detection and tracking framework for visual surveillance is proposed, which is able to handle a variable number of moving objects. Video object detectors generate an unordered set of noisy, false, missing, split, and merged measurements that make extremely complex the tracking task. Especially challenging are split detections (one object is split into several measurements) and merged detections (several objects are merged into one detection). Few approaches address this problem directly, and the existing ones use heuristics methods, or assume a known number of objects, or are not suitable for on-line applications. In this paper, a Bayesian Visual Surveillance Model is proposed that is able to manage undesirable measurements. Particularly, split and merged measurements are explicitly modeled by stochastic processes. Inference is accurately performed through a particle filtering approach that combines ancestral and MCMC sampling. Experimental results have shown a high performance of the proposed approach in real situations

Fast image decoding for block compressed sensing based encoding by using a modified smooth l0-norm

This paper proposes a fast decoding algorithm for block-based compressed sensing images that combines a modified smooth l0-norm with the BCS-SPL algorithm. Experimental results have proven a significant reduction in execution time, while providing the same image quality

Object Tracking from Unstabilized Platforms by Particle Filtering with Embedded Camera Ego Motion

Visual tracking with moving cameras is a challenging task. The global motion induced by the moving camera moves the target object outside the expected search area, according to the object dynamics. The typical approach is to use a registration algorithm to compensate the camera motion. However, in situations involving several moving objects, and backgrounds highly affected by the aperture problem, image registration quality may be very low, decreasing dramatically the performance of the tracking. In this work, a novel approach is proposed to successfully tackle the tracking with moving cameras in complex situations, which involve several independent moving objects. The key idea is to compute several hypotheses for the camera motion, instead of estimating deterministically only one. These hypotheses are combined with the object dynamics in a Particle Filter framework to predict the most probable object locations. Then, each hypothetical object location is evaluated by the measurement model using a spatiogram, which is a region descriptor based on color and spatial distributions. Experimental results show that the proposed strategy allows to accurately track an object in complex situations affected by strong ego motion

Robust automatic target tracking based on a Bayesian ego-motion compensation framework for airborne FLIR imagery

Automatic target tracking in airborne FLIR imagery is currently a challenge due to the camera ego-motion. This phenomenon distorts the spatio-temporal correlation of the video sequence, which dramatically reduces the tracking performance. Several works address this problem using ego-motion compensation strategies. They use a deterministic approach to compensate the camera motion assuming a specific model of geometric transformation. However, in real sequences a specific geometric transformation can not accurately describe the camera ego-motion for the whole sequence, and as consequence of this, the performance of the tracking stage can significantly decrease, even completely fail. The optimum transformation for each pair of consecutive frames depends on the relative depth of the elements that compose the scene, and their degree of texturization. In this work, a novel Particle Filter framework is proposed to efficiently manage several hypothesis of geometric transformations: Euclidean, affine, and projective. Each type of transformation is used to compute candidate locations of the object in the current frame. Then, each candidate is evaluated by the measurement model of the Particle Filter using the appearance information. This approach is able to adapt to different camera ego-motion conditions, and thus to satisfactorily perform the tracking. The proposed strategy has been tested on the AMCOM FLIR dataset, showing a high efficiency in the tracking of different types of targets in real working conditions

Automatic Feature-Based Stabilization of Video with Intentional Motion through a Particle Filter

Video sequences acquired by a camera mounted on a hand held device or a mobile platform are affected by unwanted shakes and jitters. In this situation, the performance of video applications, such us motion segmentation and tracking, might dramatically be decreased. Several digital video stabilization approaches have been proposed to overcome this problem. However, they are mainly based on motion estimation techniques that are prone to errors, and thus affecting the stabilization performance. On the other hand, these techniques can only obtain a successfully stabilization if the intentional camera motion is smooth, since they incorrectly filter abrupt changes in the intentional motion. In this paper a novel video stabilization technique that overcomes the aforementioned problems is presented. The motion is estimated by means of a sophisticated feature-based technique that is robust to errors, which could bias the estimation. The unwanted camera motion is filtered, while the intentional motion is successfully preserved thanks to a Particle Filter framework that is able to deal with abrupt changes in the intentional motion. The obtained results confirm the effectiveness of the proposed algorith

Learning 3D structure from 2D images using LBP features

An automatic machine learning strategy for computing the 3D structure of monocular images from a single image query using Local Binary Patterns is presented. The 3D structure is inferred through a training set composed by a repository of color and depth images, assuming that images with similar structure present similar depth maps. Local Binary Patterns are used to characterize the structure of the color images. The depth maps of those color images with a similar structure to the query image are adaptively combined and filtered to estimate the final depth map. Using public databases, promising results have been obtained outperforming other state-of-the-art algorithms and with a computational cost similar to the most efficient 2D-to-3D algorithms

Improved 2D-to-3D video conversion by fusing optical flow analysis and scene depth learning

Abstract: Automatic 2D-to-3D conversion aims to reduce the existing gap between the scarce 3D content and the incremental amount of displays that can reproduce this 3D content. Here, we present an automatic 2D-to-3D conversion algorithm that extends the functionality of the most of the existing machine learning based conversion approaches to deal with moving objects in the scene, and not only with static backgrounds. Under the assumption that images with a high similarity in color have likely a similar 3D structure, the depth of a query video sequence is inferred from a color + depth training database. First, a depth estimation for the background of each image of the query video is computed adaptively by combining the depths of the most similar images to the query ones. Then, the use of optical flow enhances the depth estimation of the different moving objects in the foreground. Promising results have been obtained in a public and widely used database